Assessing the Relationship between Hospital Process Digitalization and Hospital Quality – Evidence from Germany

Hospital digitalization aims to increase efficiency, reduce costs, and/ or improve quality of care. To assess a digitalization-quality relationship, we investigate the association between process digitalization and process and outcome quality. We use data from the German DigitalRadar (DR) project from 2021 and combine these data with two process (preoperative waiting time for osteosynthesis and hip replacement surgery after femur fracture, n = 516 and 574) and two outcome quality indicators (mortality ratio of patients hospitalized for outpatient-acquired pneumonia, n = 1,074; ratio of new decubitus cases, n = 1,519). For each indicator, we run a univariate and a multivariate regression. We measure process digitalization holistically by specifying three models with different explanatory variables: (1) the total DR-score (0 (not digitalized) to 100 (fully digitalized)), (2) the sum of DR-score sub-dimensions’ scores logically associated with an indicator, and (3) sub-dimensions’ separate scores. For the process quality indicators, all but one of the associations are insignificant. A greater DR-score is weakly associated with a lower mortality ratio of pneumonia patients (p < 0.10 in the multivariate regression). In contrast, higher process digitalization is significantly associated with a higher ratio of decubitus cases (p < 0.01 for models (1) and (2), p < 0.05 for two sub-dimensions in model (3)). Regarding decubitus, our finding might be due to better diagnosis, documentation, and reporting of decubitus cases due to digitalization rather than worse quality. Insignificant and inconclusive results might be due to the indicators’ inability to reflect quality variation and digitalization effects between hospitals. For future research, we recommend investigating within hospital effects with longitudinal data. Supplementary Information The online version contains supplementary material available at 10.1007/s10916-024-02101-y.


Background/ rationale Participants 6
Cohort study-Give the eligibility criteria, and the sources and methods of selection of participants.

Describe methods of follow-up
Case-control study-Give the eligibility criteria, and the sources and methods of case ascertainment and control selection.Give the rationale for the choice of cases and controls Cross-sectional study-Give the eligibility criteria, and the sources and methods of selection of participants Data, especially Table 1 and Figure 1 For instance paragraph starting with "All data were collected at the hospital site level.

Data -DigitalRadar Score
The DR-score is a standardized tool measuring the digital maturity of hospitals based on a questionnaire [1,2].It is scaled continuously between 0 (not digitalized) and 100 (fully digitalized).The DR-score is calculated from self-reported data which hospitals could supply via an online platform.The DigitalRadar Hospital Consortium provided a support team assisting hospitals during the data collection process.Besides, the consortium conducted plausibility and quality control of answers supplied by the participating hospitals.If necessary, hospitals were contacted to correct (add) implausible (missing) information.
In the first measurement period between October and December 2021, the survey had 234 questions.
Only 199 questions are relevant for the calculation of the score, however, as 35 questions ask for structural information such as the full-time equivalents of the IT department.Moreover, depending on the hospital type and infrastructure, not all 199 questions are relevant for the score.For psychiatric hospitals, for instance, questions relating to acute somatic care are irrelevant or for hospitals without emergency care department, questions relating to emergency care are irrelevant.Thus, for such questions, hospitals had the option to select "not relevant" as answer option.The DR-score of each hospital is normalized to the scale of 0 to 100 to allow for standardized comparisons while only considering questions relevant to a hospital's operations.For some questions, hospitals could select the answer option "Don't know" which was counted as 0.0 points.
The DR-score is structured into seven dimensions and each dimension is categorized into different subdimensions (see Table 5): Annotations: The DR dimension "structures and systems" has two more sub-dimensions "hospital information and metrics" and "IT metrics" which are not relevant for scoring.
In the DR-score dataset, dimensions and sub-dimensions are scaled between 0 and 1, representing the degree to which the total possible score of a (sub-) dimension was reached by a hospital.Thus, when summing different sub-dimensions' scores logically associated with a quality indicator, this sum is scaled between 0 and the number of summed sub-dimensions.
The total DR-score is based on the scores of the dimensions with different weights per dimension.These weights were defined by an expert advisory board including representatives from hospitals, payers, umbrella organizations of interest groups (e.g., physicians) and international digital health experts.
According to the DR interim report, the expert advisory board was continuously involved in the development of the DR-score methodology [1].
In our study, we rely on the DR-score due to two advantages compared to commonly used digitalization indicators.First, the total DR-score and its (sub-) dimensions are continuously measured.Thus, it allows for nuanced variation in its measurement.Ordinal and binary measurements, such as the EMRAM stages or EHR adoption levels, follow normative criteria to reach the next level.If not all criteria of the previous level are met, the hospital will not advance, even if some aspects of higher levels are already reached [3], limiting the measured variation and level of detail.In our view, this variation and level of detail are needed for analyzing a digitalization-quality relationship, however.Along with its (sub-) dimensions, the DR-score supplies this level of detail as well as needed variation, being almost normally distributed with the majority of hospitals in close range of the overall average (cf.the DigitalRadar Hospital Consortium interim report [1] and a publication using the DR-score for investigating a profitability-digitalization relationship [4]).Second, sub-dimensions reveal specific aspects of hospital process digitalization for which logical associations with process quality can be established (cf.section Data -Logical associations below).

Description and scale of chosen quality indicators
We use the two process quality indicators "Pre-operative waiting time before primary hip replacement surgery after fracture of the femur" (short: Preop waiting hip replacement) and "Pre-operative waiting time before osteosynthesis surgery after fracture of the femur" (short: Preop waiting osteosynthesis) for our analysis.These indicators assess the preoperative waiting time for patients with femur fractures.
Minimizing preoperative waiting time for these patients is essential both for optimal pain management as well as mortality, perioperative complications, and revisions [6][7][8].Thus, in the esQS program, a goal of less than 24 hours preoperative waiting time is set for hospitals.Both process indicators are continuous variables from 0 to 100.A value of 10, for instance, means that the requirement of a preoperative surgery time of less than 24 hours was not met in 10% of the cases [12,13].
We use the two outcome quality indicators "Risk-adjusted inpatient mortality ratio of patients hospitalized for outpatient-acquired pneumonia" and "Risk-adjusted ratio of inpatient cases with a new bedsore/ decubitus, excluding decubitus/ ulcers of level/ category 1" for our analysis.The first outcome quality indicator assesses the inpatient mortality of patients that were hospitalized for outpatientacquired pneumonia, excluding patients hospitalized with a palliative therapy goal [9].In total, 13 risk factors significantly associated with inpatient mortality are used for risk-adjustment (e.g., age, gender, chronic bed confinement, mean arterial blood pressure at admission, etc.).The second outcome quality indicator assesses the quality of nursing processes by measuring the ratio of observed to expected newly developed bedsores [10].Ten risk factors significantly associated with the development of bedsores are considered for risk-adjustment (e.g., age, number of ventilation hours, obesity, diabetes, infections, etc.).
Both outcome quality indicators are ratios of the observed over the expected number of instances.Thus, a value of 1.0 implies that exactly the number of expected instances (i.e., deaths or new cases with decubitus) were also observed.Consequentially, a ratio of less than 1.0 implies better than expected quality and generally, the lower the ratio, the better the quality of a hospital.

Reasoning for inclusion of chosen quality indicators
In the German External Inpatient Quality Assurance Program (esQS), 202 quality indicators were measured and made publicly available in 2022 [5].We selected process quality indicators according to three reasons: (1) Adequacy for testing our hypothesis regarding the influence of process digitalization on process quality and on outcome quality, (2) sufficient quality variation between hospitals, addressing condition 2 outlined in our introduction, and (3) balancing different patient groups and process areas.
Moreover, the selected quality indicators needed to be suitable for building logical associations with DR-score sub-dimensions (see next section Data -Logical Associations).
Firstly, as outlined in the introduction, we hypothesize that process digitalization might influence process quality and ultimately outcome quality.To test this hypothesis, we include two process quality indicators strongly linked to outcome quality [6][7][8].With the analysis of the two process quality indicators, we test the first part of the relationship, i.e., the direct relationship of process digitalization and process quality.We include two outcome quality indicators strongly linked to process quality [9,10] in our analysis to test the second part of our hypothesis, i.e., the indirect relationship of process digitalization and outcome quality.We chose the quality indicator "risk-adjusted inpatient mortality ratio of patients hospitalized for outpatient-acquired pneumonia" as the esQS additionally measures six process indicators for outpatient-acquired pneumonia, underscoring the strong link between process and outcome for this indication.Regarding the indicator "risk-adjusted ratio of inpatient cases with a new bedsore/ decubitus, excluding decubitus/ ulcers of level/ category 1", the main lever to avoid the development of new cases is adequate decubitus prophylaxes, typically carried out by nurses.Thus, this indicator assesses the quality of a key nursing care process.Ideally, all indicators would target the same indication and/ or procedure.Still, empirical evidence for the structure-process-outcome quality triad is scarce and it is difficult to show with secondary data (e.g., [11]).
Secondly Lastly, with our quality indicator selection we also aimed to balance surgical and non-surgical patients and (pre-) surgery, physician, and nursing focused care processes (see Table 6).

Non-surgical patients
Physician processes (e.g., complete realization of standard diagnostic measures) [9] Risk-adjusted ratio of inpatient cases with a new bedsore/ decubitus (excl.decubitus/ ulcers of level/ category 1)

All patient groups
Nursing [10] Annotations: See indicated references for more information.
We acknowledge that the disadvantage of our approach is that our findings are limited to the investigated quality indicators.Still, with our study we hope to supply a generalizable approach for investigating digitalization-quality relationships, applicable to other indicators from the esQS program and similar data collected in other countries.See hip replacement and osteosynthesis surgery after fracture of the femur above Order management An overview of all services requested within the hospital for patients is provided via the HIS.
- The hospital uses data analysis and evaluation tools to forecast medical risks for patients.Detected risks are communicated to clinical employees via automated warnings or notices (as decision support) so that they can intervene at an early stage, reduce risks, and optimize care. -

Sensitivity analysis II -Inclusion of quality indicator outliers
, we compared descriptive statistics of dozens of quality indicators, i.e., mean, standard deviation, median, 25 th and 75 th percentile, to only select indicators with relatively high variation between hospitals.The four selected indicators were among the quality indicators that showed the highest variation (cf.Results -Descriptive Statistics in the main paper).Still, as we outline in the Discussion part of the main paper, the process quality indicators might in fact not be apt to sufficiently unveil quality variation between hospitals.

( 1 )
Not implemented to (5) Fully implemented; (6) Don't know Order & medication management For employees, it is possible to control clinical work processes digitally with the help of workflows and to be automatically informed about treatment steps (status management).-(1) Not implemented to (5) Fully implemented; (6) Don't know Device & location independent flexible working Is information in the HIS available for inquiry outside the organization (i.e., remote access by clinicians)?-(1) Yes, (2) No, (3) Don't know The hospital enables clinical staff to access and process clinical information from the HIS or PDMS regardless of the device.-From (1) Not implemented to (5) Fully implemented, (6) Don'

( 4 )
Don't Know Annotations: AKTIN = Aktionsbündnis zur Verbesserung der Kommunikations-und Informationstechnologie in der Intensiv-und Notfallmedizin (Action Alliance for the Improvement of Communication and Information Technology in Intensive and Emergency Medicine); DICOM = Digital Imaging and Communications in Medicine; Docum./diagn = Documentation/ diagnosis; DR = DigitalRadar; HIS = Hospital Information System; mgt.= Management; Org.= Organizational; PDMS = Patient Data Management System;.All DR questions can be found in [2].

Table 7 : Logical matching of DR-sub-dimensions and quality indicators
Physicians in the hospital have access to medical information (e.g., initial findings, vital signs, ECGs) even before the patient arrives.This way, medical staff can, for instance, prepare themselves for a patient and organize needed resources before the patient arrives.

Table 8 : Sensitivity analysis I -matching observation period of digitalization and quality indicators
Quality data was averaged between 2020 and 2021 to increase variable robustness potentially negatively influenced by the COVID-19 pandemic.In the multivariate regression analysis, we control for federal states, bed categories, ownership, emergency level, teaching hospital status and university hospital type.Coefficients are available from the corresponding author upon request.Asterisks indicate the significance level: *** p < 0.01, ** p < 0.05, * p < 0.10.Numbers in the table indicate beta coefficients, heteroskedasticity-robust standard errors are in parentheses.

Table 9 : Sensitivity analysis II -Inclusion of quality indicator outliers
Annotations: N = number of observations.In the above analyses, we include quality indicator outliers outside of the 95%confidence interval of the sample median.In the multivariate regression analysis, we control for federal states, bed categories, ownership, emergency level, teaching hospital status and university hospital type.Coefficients are available from the corresponding author upon request.Asterisks indicate the significance level: *** p < 0.01, ** p < 0.05, * p < 0.10.Numbers in the table indicate beta coefficients, heteroskedasticity-robust standard errors are in parentheses.